Arc Post-Mortem Summary. A short follow-on to Phase 3 settling how an account authenticates and what makes a password acceptable. The Security activity's Recovery fieldset became an Authentication fieldset: pick the sign-in method, set the password rules, configure LDAP in the panel instead of LocalConfig.php, and choose recovery (forced off under LDAP). Other work was folded in as it surfaced.
Arc started 2026-06-25, a short follow-on to Phase 3. Phase 3
settled how an account recovers a password; this arc settles how an
account authenticates and what makes a password acceptable. The Security
activity's “Recovery” fieldset becomes an
“Authentication” fieldset: pick the sign-in method, set the
password rules, configure LDAP in the panel instead of
LocalConfig.php, and choose recovery (forced off under LDAP).
Marker legend: ✓ done,
▶ next, ? decision
to make, unmarked = planned.
SecurityElement (with
options from SystemComponent) the fieldset's first control
chooses Yioop authentication or LDAP.
The rest of the fieldset shows or hides on that choice the way the Mail
Services controls already do: Yioop → the password-rule controls
(item 2); LDAP → the directory settings (item 4).
AUTH_METHOD profile setting (constants
LOCAL_AUTHENTICATION / LDAP_AUTHENTICATION,
defaulting to local); the recovery dropdown sits under an Account
Recovery label and is hidden when LDAP is chosen: the dropdown's
onchange calls the existing toggleClass helper
to add or remove the none class, and the section's initial
hidden state is a conditional class rendered in PHP, so there is no
inline style or script. Sign-in behaviour is unchanged for now —
AUTH_METHOD is read by the sign-in path in items 4-5.
AUTH_METHOD in ProfileModel::profile_fields
and profile_defaults (without this the setting never wrote
to Profile.php and a saved choice was lost on reload);
renamed the constant and label from Yioop authentication to the
layperson term Locally Stored Passwords; and added colons to the
Sign-in Method and Account Recovery labels.RECOVERY_MODE.
PASSWORD_MIN_LEN and four
PASSWORD_REQUIRE_* booleans) with Config defaults,
registered in ProfileModel, saved by the Security
activity, and rendered as four checkboxes plus a minimum-length field.
The password rules and the Account Recovery control now share one
local-auth-section div that the method dropdown shows for
Locally Stored Passwords and hides for LDAP (item 4 adds the opposite
LDAP section). The minimum length is clamped server-side to
1–LONG_NAME_LEN. Shipped defaults: minimum
length 8 and no character-class requirement checked, so upgrading does
not make any existing member's password suddenly invalid; an operator
opts into stricter rules. The controls only store policy here —
item 3 enforces it in the validator and on the server.register_validator.js and its unit test. Make the check
shared, enforce the item-2 rules on both sides (the JS and a single
server-side check so the two never disagree and JS-off cannot store a
weak password), and wire it into Create Account, Manage Account →
Change Password, the Change Recovery Question confirm field, both
Recover-Account new-password screens, and the admin Manage-Users
add/edit password. The JS is updated and improved while staying plain
self-contained JS (no outside library).
L\passwordPolicyViolations($password) in
Utility.php — it reads the item-2 profile rules plus
a new PASSWORD_FORBIDDEN_CHARS constant (the single and
double quote by default, since an unescaped quote can break a form value
or a query) and returns a list of unmet-rule tokens, with a unit test
over the shipped defaults. It is enforced server-side on every path that
sets a password: the three member self-service paths (Create Account,
Manage Account → Change Password, Recover-Account) and now both
admin Manage-Users paths (add-user, and edit-user when an administrator
actually changes the password — that path overloads one error flag
for "passwords do not match", so the rule check gets its own message).
On the client the same check now lives in the long-standing
register_validator.js (its own hard-coded strength regular
expressions are gone), generalised so its create-account-only field
wiring is guarded and the script is safe to load on every password form.
A new PasswordrequirementsHelper draws, next to each
password box, a plain-English sentence listing exactly the rules in
force (smallest length, whichever character classes the operator turned
on, and the forbidden characters), and carries those rules — plus
the per-rule wording — to the script as plain data attributes.
When JavaScript is running the script replaces that sentence with live
colored feedback: green and a short "looks good" once the password meets
every rule, or red naming just the requirements still missing; it also
blocks submitting an unacceptable password. When JavaScript is off the
sentence simply stays visible so the rules are still known. The same
feedback now runs on all the password forms (Create Account, Change
Password, Recover-Account, and the admin Manage-Users add/edit boxes),
which load the shared script. The two server flash messages were
shortened to "Password requirements not met." since the on-screen hint
now spells the rules out.LocalConfig.php into the
panel. When the method is LDAP, the fieldset offers fields for
LDAP_ACCOUNT_SUFFIX, LDAP_BASE_DN, and the
server host (today LDAP_CONTROLLERS). The
LDAP_LOCAL_USER callback becomes a real
SigninModel method: given an email, return the Yioop
username that owns it, or false. Because LDAP keys members by email,
two accounts may not share an email once LDAP is in use; add that
constraint and a pre-switch check.
Done. The three settings are now profile fields with
Config defaults, registered in ProfileModel,
saved by the Security activity, and shown in an ldap-section
the sign-in-method dropdown reveals for LDAP the same way it hides the
local-password section. The sign-in path now gates on
AUTH_METHOD rather than the old USE_LDAP
constant, reads the three profile settings, and maps the email the
directory returns through the new
SigninModel::localUserForLdapEmail(), which looks the
email up in the accounts table and returns the owning username, or
false when no account or more than one has it. Email uniqueness is kept
while LDAP is on: changeEmail refuses an email another
account already holds, and a pre-switch scan
(SigninModel::emailConflicts()) blocks turning LDAP on
while any email is shared. The three decisions were settled as:
the host becomes a panel field, added and removed with the same list
control as SECURE_DOMAINS in Server Settings (a reusable
initManagedCsvList in basic.js keeps a hidden
comma-separated value in step); the LDAP_USER_LIST map is
dropped in favour of the accounts-table lookup; and a blocked switch
flashes that accounts share an email and offers a link in the panel
that downloads a plain-text file listing each shared email and the
usernames that share it.
Follow-up (pending commit): the directory-server list now matches the
Server Settings control exactly — an "+" add button rather
than the word, no placeholder text, and the shared list styling —
and its wiring is run from window.onload so the add button
works (it had been called before the page scripts loaded, leaving the
button dead).
Superseded by item 5: the original blocking behaviour here —
refusing the switch and reverting to local when emails clashed —
was replaced by the readiness gate. The switch now always saves; a
shared email is just one of the things that keeps LDAP from becoming
active, reported in the panel's persistent problem list (still with the
downloadable clash file, whose first row is now an
Email/Usernames header) rather than as a
one-off refusal.SigninModel::ldapConfigProblems() returns what is
left to fix — no directory server, a missing account suffix or
base DN, an email shared by two accounts, or a root account with no
email (so an LDAP sign-in can always reach the administrator) —
the sign-in path falls back to local passwords whenever that list is
non-empty, and the Authentication panel shows the same list
persistently with an asterisk on each needed field. The actual proof
that root can bind then comes from root's first real LDAP sign-in,
which the operator can attempt while still holding a working local
sign-in.setRecoveryForLdap helper as the method changes, and the
Security activity stores NO_RECOVERY whatever the form
sent so a disabled control can never leave a stale recovery mode in
place. This completes Phase 3.75.
Follow-up (pending commit): a disabled control submits nothing, so
under LDAP the recovery dropdown left RECOVERY_MODE out of
the request and the Security activity warned twice on the undefined
key. Rather than have the server skip a missing field, the form now
always sends the value: a hidden companion field carries the off
setting while the dropdown is disabled, and the sign-in-method change
swaps which of the two is active so exactly one submits
RECOVERY_MODE.sudo php index.php
secure-restart failed to come back up, the replacement server
logging Address already in use until it gave up after 120
seconds. Background daemons are launched through popen,
and a process launched that way inherits every open descriptor of its
launcher — including the web server's listening sockets —
so MediaUpdater kept its own copy of ports 80 and 443 and held them in
use until it exited, which with a long job in flight was minutes away.
A standalone test reproduced it exactly. There is no portable way to
mark a PHP stream socket close-on-exec, and closing the numbers from
inside the launched program does not release them, so the launch
command now closes every inherited descriptor above standard error in
the launching shell before the program starts; the command building
moved into a pure composeBackgroundJob so it can be
checked directly (CrawlDaemonTest). The closing applies on
the shell-launched path mac and linux share; Windows starts the child
without handing it the socket handles, so its launch line is
unchanged. MediaUpdater also now checks for a stop request before each
job rather than only between whole passes, so a stop is noticed without
finishing the rest of the list. A single job that is itself long is
interrupted too: MediaUpdater installs a stop-signal handler, and a stop
of MediaUpdater sends that signal, so the running job is cut off and the
daemon ends at once. This is general across every MediaUpdater job, not
a per-job change, and is limited to daemons whose work is safe to cut
off (an interrupted mail batch is gathered again on the next pass), so
index-writing daemons are left to stop at their own checkpoints. Where
the signal functions are absent, for example on Windows, the
between-jobs check stays the fallback.UpgradeFunctions.php's one-row version check, but that
check cannot need the memory it failed to get, so the landing spot is
incidental: the process was already near the limit and that small
request was simply the one that tipped it over. A first pass ruled out
the obvious holders: the persistent server already releases every
per-connection and per-request item it keeps — open streams,
timers, the connection record, and the saved request context are all
cleared when a connection closes or an internal request returns —
and the request-path caches that were checked
(UrlParser's site-list cache,
IndexManager's url cache) each have a size cap. The
leftover suspect is something kept in a long-lived place that survives
from one request to the next and is added to per request without a cap;
such a thing is harmless under the one-request-per-process model but
accumulates in the always-on server. A single crash trace will not
point to it, because the gradual grower is not the request that finally
fails; finding it needs a memory-and-connection-count reading sampled
over time as the server runs under load. An error log is now set to
capture the next occurrence, and a switchable memory probe has been
added to the website server to go with it. When
WEBSITE_MEMORY_SAMPLE_PERIOD is set to a positive number of
seconds — for example by adding
nsconddefine("WEBSITE_MEMORY_SAMPLE_PERIOD", 30);
to src/configs/LocalConfig.php — the server writes
one line to the error log that often, giving current and peak memory in
megabytes alongside the live counts of connections, reading streams,
writing streams, sessions and timers. Zero or absent leaves it off, and
the per-pass timing check was measured to cost nothing noticeable.
Reading those lines over a few hours should show whether memory climbs
while the connection and stream counts stay flat, which would point at
something held across requests rather than a connection that is never
let go, and narrow down where to look next.MailSiteTest with a single test class. Because the
groups had used the same property names for different things (the
probe was four different kinds; the mail server and storage were
built two ways), each group's set-up became a small helper its
own cases call first, so nothing collides and every case keeps its
original behaviour; all twenty-six pass. The same was done for
MailSiteFactory, whose address-permission rule and
outbound mail spool were in two files and are now one
MailSiteFactoryTest of twelve cases. Tests that look
related by name but cover their own class are left alone: the six
Imap parser and client tests, MailSiteMailBackend,
and the cooperative scheduler test, which borrows a website-shaped
probe but tests the scheduler.SearchController made seven
webExit() calls that resolved to
seekquarry\yioop\library\webExit (five written out that
way in full, two through the library alias) instead of the atto
webExit that actually exists, so each of those paths died
with “call to undefined function” — the deferred-search
path among them. They now call the atto webExit, matching
the one already-correct call in the same file. This shows up on
yioop.com because it is the search-engine install and leans on
SearchController heavily, where the wiki sites rarely reach
that path. Second, a stray bare header: the WebSocket-upgrade
“501 Not Implemented” reply in WebSite::process
called PHP's own header() rather than the server's
header method. In the always-on server the one-time startup banner has
already begun output, so a bare header() warns
“headers already sent” and the status never reaches the
client. It now goes through the WebSite header method, which buffers to
the socket in CLI mode and falls back to PHP's header()
only in SAPI mode — the single place that fallback belongs.
PHP's header() is never to be called directly in
WebSite or Yioop code.secure-terminal run was stopped with Ctrl-C it would not
start again until a separate secure-stop was run first,
and on yioop.com (where the out-of-memory fatal had killed the server)
secure-start quietly started nothing. Both came from how a
running server is recognized. The start refusal leaned on a fifteen-
minute timestamp window, and a lock written without the server's
process id reads as “still running” because the liveness
check cannot tell — so a fresh lock left behind by a server that
had just died looked alive and blocked a new start for up to fifteen
minutes. Now every lock write records the process id, and a new
lockHeldByLiveProcess decides held-or-not from that id
directly: a server that crashed or was Ctrl-C'd is seen as gone at
once, with the timestamp window kept only as a fallback for older locks
or hosts without the POSIX calls. That fallback window was still the
full fifteen-minute PROCESS_TIMEOUT, far too long: an
id-less lock left by a server that crashed before this change (the one
stuck on yioop.com among them) read as alive and blocked the next start
for up to a quarter hour. It is now a short one-minute
STALE_LOCK_TIMEOUT, so an id-less stale lock clears within
a minute. A start or terminal run that finds a
stale lock removes it and proceeds; one that finds a genuinely live
server prints a clear “already running” message and stops,
so a refusal is never silent. The terminal run now makes this check
too, so a Ctrl-C'd server's leftover lock no longer needs a
separate stop. The same recorded id lets the status report drop a
crashed daemon immediately rather than after fifteen minutes.php index.php start and
secure-start would print their startup lines and then
leave nothing running — the web server never came up, with no
error shown. The child was healthy when run in the foreground, so the
launch path itself was at fault. The blanket close walked descriptors
3 upward and closed the ones the launching shell relies on, so on the
dash shell that is /bin/sh on many Linux installs the
shell died part way through the loop and never reached the step that
starts the program; macOS, whose /bin/sh is bash, did not
trip on it, which is why only Linux broke. The descriptor-closing loop
has been removed and the launch is the plain background start it was
before. Keeping a restarted server from inheriting the old socket is a
separate task to solve at the socket itself rather than by closing
descriptors blindly in the launch line.get_resources lists them. The launch now forks, and the
forked child closes the inherited listening sockets — then becomes
the program through a shell that reads nothing from input and sends
output to the requested file. Two cautions learned on the live
Postgres server, neither visible against the laptop's SQLite.
First, only listening sockets are closed, never connected
ones: a socket with no peer is a listener and safe to drop, but
fclose on an inherited connected link (a request in
flight, or the database connection) sends a shutdown notice on the
socket the launching server still shares, which on yioop.com closed the
server's own Postgres link mid-spawn and every following query
failed with SSL connection has been closed unexpectedly.
Second, the brief helper processes the two forks create are ended with
an immediate kill rather than a normal exit, because a normal exit runs
PHP's shutdown and the inherited database handle's destructor
would close that same shared connection. Closing the three standard
streams from inside PHP was also tried and quietly broke the index
server (it exited inside its listen call with no message because its own
error stream had been closed), so the shell does the redirect instead.
Two forks with a new session in the middle let the launcher return at
once and leave no defunct entry behind. Where the process-control
functions are absent (Windows, or a build without pcntl)
the plain background launch stays the fallback. A standalone test holds
a listener and a connected socket, forks a worker, and confirms the
listener is dropped while the connected socket keeps working and a fresh
process rebinds the freed port; start, stop, and restart each come up
and serve.